1. Field of the Invention
The present invention relates to an image processing apparatus and an image processing method used for correcting a camera shake to prevent a blur of a moving image captured by a video camera or the like.
2. Description of the Related Art
For users of an imaging apparatus such as a video camera used for capturing a moving image, a blurred image that is generated due to a camera shake, especially when the focus lens is zoomed at a telephoto end is a problem.
To prevent such an image blur due to camera shake, there has conventionally been proposed a technique that detects a motion vector of an image from a captured image signal and corrects the image blur based on the detected motion vector. An imaging apparatus that uses this technique is discussed in Japanese Patent Application Laid-Open No. 06-133298.
Further, a correlation method based on a correlation operation and a block matching method are conventionally used as methods for detecting a motion vector of an image.
According to the block matching, an input image signal is divided into a plurality of blocks (hereinafter referred to as vector detection areas) each having an appropriate area size such as eight pixels×eight lines. A difference between pixels in a predetermined range in a preceding field (or frame) and in the current field is calculated for each vector detection area. A vector detection area of the preceding field (or frame) in which the sum of the absolute value of the difference becomes minimum is searched for. The relative shift between the fields represents the motion vector of the vector detection area.
Matching calculation is discussed in detail by Morio Onoe, et al., in “Information Processing”, Vol. 17, No. 7, pp. 634-640, July 1976.
An example of a conventional motion vector detection method using the block matching will now be described referring to FIG. 15. FIG. 15 is a schematic block diagram of an apparatus that prevents a blur using a conventional motion vector detection method.
An image signal (field or frame) subjected to motion vector detection is input to an image memory 101 and a filter 102 used for extracting spatial frequency. The image memory 101 temporarily stores the image signal.
The filter 102 extracts, from the image signal, a spatial frequency component useful for motion vector detection. In other words, the filter 102 removes the low and high spatial frequency components of the image signal.
The image signal that passed through the filter 102 is input to a binarization circuit 103. The binarization circuit 103 binarizes the image signal using zero level as a reference. More specifically, the binarization circuit 103 outputs a sign bit of the output signal.
The binary image signal is input to a correlation calculation circuit 104 and a memory 105 serving as a 1-field period delay unit. The correlation calculation circuit 104 further receives an image signal of a preceding field from the memory 105.
According to the block matching, the image area is divided into a vector detection area of a convenient size, and the correlation calculation circuit 104 calculates the correlation between the temporally continuous image data for each vector detection area. The resultant correlation value is output to a motion vector detection circuit 106.
The motion vector detection circuit 106 detects a motion vector for each vector detection area from the calculated correlation value. More specifically, the motion vector detection circuit 106 searches for a vector detection area of a preceding field having a minimum correlation value. Then, the motion vector detection circuit 106 detects a relative shift between the motion vector detection areas of the current and the preceding fields as a motion vector.
The motion vector of each vector detection area is input to a motion vector determination circuit 107. The motion vector determination circuit 107 determines a representative motion vector, which represents a motion vector of a whole screen, from the motion vector of each of the vector detection areas. More specifically, the motion vector determination circuit 107 determines the median or average of the motion vectors of the vector detection areas as the motion vector of the entire image.
The motion vector determination circuit 107 outputs the motion vector of the entire image to a memory read control circuit 108. The memory read control circuit 108 controls the read position in the image memory 101 so as to cancel image shift in accordance with the motion vector of the entire image. Then, a blurring-corrected image signal is output from the image memory 101.
However, according to the above-described method, if a moving object is included in the image, for example, if an average of the detected motion vectors is determined as the motion vector of the entire image, movement of the moving object in the image will be mixed with the movement of the image.
As a result, the reading position of the image memory changes such that the moving object remains at its original position in the screen. Accordingly, a position in an area in the screen, which does not move under normal circumstances, changes. Thus, the screen appears as though it is drawn by the moving object.
In order to prevent this unnatural motion, it is necessary to control so that the screen only follows the motion vectors of the background and not the areas of the moving object.
Japanese Patent Application Laid-Open No. 2007-235769 discusses a method for preventing a blur when a moving object exists in the screen. According to this method, a vector group that occupies a great number of the detected areas is detected, and the background and the moving object are identified using the vector group.
However, since the method discussed in Japanese Patent Application Laid-Open No. 2007-235769 uses information of a discrete degree of the vector areas in the group for each single frame in determining the area of the moving object, the moving object area determination is not associated in time sequence.
Thus, if the discrete degrees of the background and the moving object are similar, or if the moving object gradually becomes larger, determination error of the area that needs correction of a blurred image due to camera shake or a group that needs to have a blurred image corrected occurs.